Method and System to Combine Keyword and Natural Language Search Results

ABSTRACT

A search query is received from a single input field of a user interface. A keyword search is performed based on the search query to generate keyword search results. A natural language search is performed of a frequently-asked question (FAQ) database based on the search query to generate FAQ search results. The keyword search results and the FAQ search results are combined in a display page.

CLAIM OF PRIORITY

The present application claims priority from and is a continuation ofU.S. patent application Ser. No. 12/636,716, filed on Dec. 12, 2009 andtitled “METHOD AND SYSTEM TO COMBINE KEYWORD RESULTS AND NATURALLANGUAGE SEARCH RESULTS,” which claims priority from and is acontinuation of U.S. patent application Ser. No. 10/975,023, filed onOct. 27, 2004, now issued as U.S. Pat. No. 7,668,889, and titled “METHODAND SYSTEM TO COMBINE KEYWORD AND NATURAL LANGUAGE SEARCH RESULTS,” thecontents of each of which are expressly incorporated herein by referencein their entirety.

BACKGROUND

A challenge for a developer of a Web site is predicting what users willwant to do on the Web site and whether all users will understand thenames of the links provided on the Web site. Since some users prefer tonavigate a Web site by typing what they are looking for, many Web sitesprovide a search engine to allow users to find what they are looking forusing their own terminology. Allowing users to find what they arelooking for quickly and easily often translates into increased salesand/or reduced support costs.

The development of natural language search algorithms has made use ofnatural language searching of Web sites more feasible and popular. Anatural language search engine receives full sentences (which may be inEnglish or another language) as typed or otherwise entered by users, andsearches a set of data based on both the words in the sentence and theconstruction of the sentence. Natural language search engines are oftenused to retrieve answers to frequently asked questions (FAQs) bysearching a FAQ database including a number of questions and answers.

Keyword search engines are also provided by many Web sites. A keywordsearch engine searches an entire Web site for pages and other resourcesrelated to user-entered keywords.

When both a natural-language FAQ search feature and a keyword searchfeature are provided by a Web site, the user may become confused as towhich search engine to use. Further, the placement and naming of the twosearch features can become problematic for a developer of the Web site.

Further, search results are generally presented in a linear list withmultiple types of results being intermixed. Some of the search resultsmay have little bearing on the user's intended search. For example, auser searching on “DSL” may be presented search results related to salesof DSL, technical aspects of DSL, technical support for DSL and otherDSL-related results, all mixed together without preference given to theuser's area of focus. Recent research has pointed to the use ofcategorization of search results as a method to allow users to morequickly and easily find the results they are looking for.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is pointed out with particularity in the appendedclaims. However, other features are described in the following detaileddescription in conjunction with the accompanying drawings in which:

FIG. 1 is a flow chart of an embodiment of a method of providingcombined search results;

FIG. 2 is a block diagram of an embodiment of a system for providingcombined search results;

FIG. 3 is a screen shot of an embodiment of the user interface in FIG.2;

FIG. 4 is a screen shot of an embodiment of the first display page inFIG. 2;

FIG. 5 is a screen shot of an embodiment of the second display page inFIG. 2;

FIG. 6 is a screen shot of an embodiment of the first display page inFIG. 2 that includes keyword search results but not FAQ search results;and

FIG. 7 is a screen shot of an example of a detailed search results pagethat includes a drop-down menu for modifying a category.

DETAILED DESCRIPTION

Tests have shown that about half of sampled users prefer keywordsearches, and about another half prefer natural language searches insome applications. To satisfy all users, embodiments of the presentdisclosure perform both types of searches based on a search queryentered into a single input field, and combine the two resulting typesof search results into a single display page.

To assist users in focusing on particular areas of interest in thesearch results, the display page categorizes the keyword search resultsand the FAQ search results into a plurality of categories. The types andnumber of categories are determined using a card sort technique. In theabove-described DSL example, the display page may categorize the DSLsearch results into products and services category, a billing andaccount category, a help and support category, and a corporateinformation category. The user can view and directly expand on the helpand support category, for example, from the display page to retrieve theinformation he/she is looking for.

Embodiments of the present disclosure are described with reference toFIG. 1, which is a flow chart of an embodiment of a method of providingcombined search results, and FIG. 2, which is a block diagram of anembodiment of a system to provide combined search results.

As indicated by block 10, the method comprises receiving a search query12 from a single input field 14 of a user interface 16. The userinterface 16 may be provided by a search service that provides searchresults for multiple Web sites or by a page of a Web site that providessearch results for other pages of the Web site. The user interface 16may accompany the single input field with a textual display to clearlyindicate to users that they may enter search queries that compriseeither keyword(s) or a full question. In one embodiment, the textualdisplay accompanying the single input field comprises “search or ask aquestion”.

As indicated by block 20, the method comprises performing a keywordsearch based on the search query 12 to generate keyword search results22. The keyword search is performed by a keyword search engine 24. Thekeyword search engine 24 performs the keyword search of contents of oneor more Web sites 26. If the user interface 16 is provided by a generalpurpose search service, the contents of one or more Web sites 26comprise contents from multiple Web sites that have been indexed by thesearch service. If the user interface 16 is provided by a Web page toenable users to search within a single Web site, the contents of one ormore Web sites 26 may comprise content only from multiple pages of thesame Web site.

As indicated by block 30, the method comprises performing a naturallanguage search based on the search query 12 to generate search results32. The natural language search is performed by a natural languagesearch engine 34. The natural language search engine 34 performs thenatural language search of contents of a frequently-asked questions(FAQ) database 36. If the user interface 16 is provided by a Web page toenable users to search within a single Web site, the FAQ database 36 maycomprise questions and associated answers that are frequently asked oranticipated to be asked by users of the Web site.

As indicated by block 40, the method comprises combining the keywordsearch results 22 and the FAQ search results 32 in a display page 44.

In one embodiment, the display page 44 is divided into two columns: afirst column to contain some or all of the keyword search results 22 butnone of the FAQ search results 32, and a second column to contain someor all of the FAQ search results 32 but none of the keyword searchresults 22.

Optionally, the display page 44 categorizes the keyword search results22 and the FAQ search results into a plurality of categories. Forpurposes of illustration and example, the display page 44 is shown tohave three different categories A, B and C, although any number ofcategories may be used. In this example, the keyword search results 22are divided into keyword search results 50 in category A, keyword searchresults 52 in category B, and keyword search results 54 in category C.Similarly, the FAQ search results 32 are divided into FAQ search results60 in category A, FAQ search results 62 in category B, and FAQ searchresults 64 in category C.

The particular categories A, B and C are application-dependent. In someapplications, the categories may include products and services category,a billing and account category, a help and support category, and acorporate information category. These categories may be of particularinterest when the user interface 16 is part of a telecommunicationservice provider's Web site.

The categories may be determined using a card sort method prior toreceiving the search query 12. The card sort method comprises creating arepresentative sample of some or all of the Web site content on cuecards. The cue cards may be either physical cards or virtual cardsdisplayed using a computer. A first sample of individuals organize thecue cards into groups and name the groups in a manner that makes senseto them. The individuals may perform these acts either individually orcollectively. If performed individually, a statistical cluster analysisis performed to determine an optimal set of groupings that accommodatesas many of the individuals' groupings as possible. The naming of thegroups in the optimal set are based on the names provided by the firstsample of individuals. The naming and nature of the groupings arevalidated using a reverse card sort. The reverse card sort compriseshaving a second sample of individuals, different from the individuals inthe first sample, sort the cue cards into the defined groups. Unlikeexisting search engines that use company created categories, the cardsort method generates search categories that match users' mental models.

In general, the layout of the search results on the display page may bedetermined by performing one or more usability tests prior to receivingthe search query 12. Based on the results of a usability test for alayout, a modified layout may be determined and subjected to asubsequent usability test. This process may be repeated until adesirable layout is achieved.

In one embodiment of the layout, the display page 44 is divided intorows, where each row is dedicated to display keyword search results andFAQ search results for an associated one of the categories. For example,the keyword search results 50 in category A and the FAQ search results60 in category A are displayed in a first row dedicated to category A.The keyword search results 52 in category B and the FAQ search results62 in category B are displayed in a second row dedicated to category B.The keyword search results 54 in category C and the FAQ search results64 in category C are displayed in a third row dedicated to category C.

The display page 44 may display all of the keyword search results and/orFAQ search results in a particular category, or may display only asubset of the keyword search results and/or FAQ search results in aparticular category. In one embodiment, at most three keyword searchresults and at most three FAQ search results are displayed in eachparticular category. In this embodiment, the top one, two or threekeyword search results and the top one, two or three FAQ search results(based on the search query 12) may be displayed in each particularcategory.

Presenting the top three results for each category directly on the firstdisplay page 44 promotes user understanding of the nature of eachcategory based on both the displayed name of the category and the searchresults in the category. Further, the user is allowed to find links tothe particular result(s) of interest to him/her directly from the firstdisplay page 44. These features distinguish from existing search enginesthat show only company-created category names on the front page thatrequire users to guess the meaning of each category and to go to asubsequent page to see the results of a category.

The order of the categories in the display page 44 may be either fixedor variable. Optionally, the order of the categories varies based on howwell the results in each category match the search query 12. In oneembodiment, the category whose results best match the search query 12 isdisplayed in the first row, followed in the second row by the categorywhose results have the second best match with the search query 12, andso on. For example, if the search query 12 is “slow DSL”, the “help andsupport” category may be presented first in a list of categories becausethe best matches for “slow DSL” are within the “help and support”category. When viewed from top to bottom, the display page 44 displayscategories with decreasingly better matches in this embodiment.

Optionally, the display page 44 also includes the user interface 16 sothat the user can refine his/her search or enter a different searchquery.

As indicated by block 66, the method optionally comprises including inthe display page 44 a user-selectable command to expand a number ofkeyword search results and/or FAQ search results that are displayedtogether in a category. In the example of FIG. 2, the display page 44may further comprise user-selectable commands 70, 72 and 74 to expand anumber of keyword search results and FAQ search results that aredisplayed together in categories A, B or C, respectively.

As indicated by block 76, the method comprises outputting a signal todisplay the display page 44 to the user who entered the search query 12.The display page 44 may be encoded in a markup language such asHypertext Markup Language (HTML) or an alternative type of computercode.

As indicated by block 80, the method optionally comprises outputtinganother display page 82 that expands a number of keyword search resultsand FAQ search results that are displayed together in a category inresponse to a user selection of one of the commands 70, 72 or 74. Forpurposes of illustration and example, consider the user selecting thecommand 74. In response thereto, expanded keyword search results 84 incategory C and expanded FAQ search results 86 in category C aredisplayed in the display page 82. In one embodiment, the display page 82includes ten keyword search results and ten FAQ search results from theuser-selected category.

Optionally, each of the display pages 44 and 82 includes a header 90 and92, respectively, having a user-selectable option to view questions in auser-selected one of the categories. The questions may comprise the mostfrequently asked and/or most relevant questions from the FAQ database 36that are in the user-selected category. The header 90 and 92 provides analternative for browsing the questions and answers in the FAQ database36 if, for example, the user is having difficulty with the search orwants to see what other related information is in the FAQ database 36.

Optionally, the display page 82 also includes a user-selectabledrop-down menu (such as a drop-down box, for example) with the othercategories so that users may view and expand the view of their searchresults in another category without having to return to the display page44 and select another option (such as the command 70 or the command 72,for example).

Optionally, the display page 82 also includes the user interface 16 sothat the user can refine his/her search or enter a different searchquery.

The herein-disclosed system can be embodied by a computer systemcomprising one or more computers. The acts performed by each of theherein-disclosed components can be directed by respective computerprogram code embodied in a computer-readable form on a computer-readablemedium. Each of the herein-disclosed components may comprise arespective computer processor responsive to the computer program code toperform the acts.

FIG. 3 is a screen shot of a user interface 16′ that is an embodiment ofthe user interface 16 in FIG. 2. The user interface 16′ has a singleentry field 14′ accompanied by an instruction to the user to “search orask a question”. Based on the instruction, users clearly recognize thatthey may use the single entry field 14′ to type either one or morekeywords or a full question, thus eliminating a need for users to choosewhich search tool to use. For purposes of illustration and example,consider a user entering a search query of “long distance” into thesingle entry field 14′. The keyword search engine 24 searches thecontents of a telecommunication service provider's Web site to find Webpages related to “long distance”. The natural language search engine 34searches the FAQ database 36 to find questions and/or answers related to“long distance”.

FIG. 4 is a screen shot of a display page 44′ that is an embodiment ofthe display page 44 in FIG. 2. The display page 44′ is a search resultspage that combines keyword search results (e.g. the Web pages related to“long distance” in the telecommunication service provider's Web site)and FAQ search results (e.g. questions and/or answers from the FAQdatabase 36 that are related to “long distance”). Presenting both setsof results on the first page increases the likelihood that the user willfind what he/she is looking for without having to navigate to otherpages of search results, and reduces a number of pages to be maintainedby developers.

The display page 44′ is divided into rows and columns to distinguishbetween different categories of search results (e.g. products andservices, billing and account, and help and support) and different typesof search results (e.g. keyword and FAQ results), respectively.Hyperlinks 70′, 72′ and 74′ are embodiments of the commands 70, 72 and74, respectively, to expand the search results in a respective category.For purposes of illustration and example, consider the user clicking onor otherwise selecting the hyperlink 70′ for more products and servicesresults.

FIG. 5 is a screen shot of a display page 82′ that is an embodiment ofthe display page 82 in FIG. 2. The display page 82′ is displayed to theuser in response to the selection of the hyperlink 70′. The display page82′ lists an expanded number of keyword search results and an expandednumber of FAQ search results in the products and services category forthe search query of “long distance”.

With reference to FIGS. 4 and 5, the display pages 44′ and 82′ includeheaders 90′ and 92′, respectively, to view top questions for allcategories, or for a particular category selected using a drop-down box.

Some of the herein-disclosed features may be applied to provide a searchresults user interface that does not combine keyword search results andFAQ search results. For example, FIG. 6 is a screen shot of a displaypage 44″ that is an embodiment of the display page 44 in FIG. 2 thatincludes keyword search results but not FAQ search results. For a searchquery of “DSL”, the display page 44″ provides DSL-related search resultscategorized into products and services results, billing and accountresults, help and support results and corporate information results.Links to the top three DSL-related results for each category aredisplayed, as well as links to get more results from any category. If auser wants to view more results for a section, he/she is taken to asubsequent results page with ten results per page. The subsequentresults page enables the user to modify the category that he/she wishesto view in detail at any time using a drop-down menu at or near the topof the page.

FIG. 7 is a screen shot of an example of a detailed search results page96 that includes a drop-down menu 98 for modifying the category. Forpurposes of illustration and example, the page 96 is generated for auser-entered search query of “call forwarding” and for a user selectionof the help and customer support category. The drop-down menu 98 enablesthe user to jump from the page 96 directly to either a second detailedpage dedicated to call-forwarding-related search results in the productsand services category, a third detailed page dedicated tocall-forwarding-related search results in the billing and accountcategory, or a fourth detailed page dedicated to call-forwarding-relatedsearch results in the corporate information category. This featuredistinguishes over existing search engines that require the user to goback to a main results page to select another category.

It will be apparent to those skilled in the art that the disclosedembodiments may be modified in numerous ways and may assume manyembodiments other than the particular forms specifically set out anddescribed herein.

The above disclosed subject matter is to be considered illustrative, andnot restrictive, and the appended claims are intended to cover all suchmodifications, enhancements, and other embodiments which fall within thescope of the present invention. Thus, to the maximum extent allowed bylaw, the scope of the present invention is to be determined by thebroadest permissible interpretation of the following claims and theirequivalents, and shall not be restricted or limited by the foregoingdetailed description.

What is claimed is:
 1. A method, comprising: receiving a search queryfor a particular application from an input field of a user interface;performing a keyword search at a computer device based on the searchquery to generate keyword search results; performing a natural languagesearch of a frequently-asked question database based on the search queryto generate frequently-asked question search results; and outputting adisplay page, wherein the display page categorizes the keyword searchresults and the frequently-asked question search results into aplurality of categories, wherein each category of the display page isassociated with a first display region and a second display region thatis separate from the first display region, wherein the first displayregion of a particular category includes one or more keyword searchresults associated with the particular category and does not include anyof the frequently-asked question search results, and wherein the seconddisplay region associated with the particular category includes one ormore frequently-asked question search results associated with theparticular category and does not include any of the keyword searchresults; wherein the plurality of categories are based on groupassignments and name assignments; and wherein the group assignments andthe name assignments are determined using a card sort method.
 2. Themethod of claim 1, wherein the group assignments and the nameassignments are validated via a reverse card sort method.
 3. The methodof claim 2, wherein the group assignments and the name assignments areperformed by a first set of individuals.
 4. The method of claim 3,wherein the reverse card sort method is performed by a second set ofindividuals.
 5. The method of claim 1, wherein an order of thecategories shown in the display page is a fixed order.
 6. The method ofclaim 1, wherein an order of the categories shown in the display page isvariable based on how well results in each category match the searchquery.
 7. The method of claim 1, wherein the display page includes asecond user interface that enables refinement of the search query. 8.The method of claim 1, wherein the display page includes auser-selectable option to output a second display page that displaysadditional keyword search results and additional frequently-askedquestion search results.
 9. The method of claim 8, wherein the keywordsearch results of the second display page are displayed in a firstcolumn and the frequently-asked question search results of the seconddisplay page are displayed in a second column.
 10. The method of claim1, wherein the first display region associated with the particularcategory is a first column and wherein the second display regionassociated with the particular category is a second column adjacent tothe first column.
 11. The method of claim 10, wherein a keyword searchresult of the one or more keyword search results associated with theparticular category that best matches the search query is in a first rowof the first column.
 12. A computer-readable storage device comprisinginstructions that, when executed by a processor, cause the processor toperform operations comprising: receiving a search query for a particularapplication from an input field of a user interface; performing akeyword search based on the search query to generate keyword searchresults; performing a natural language search of a frequently-askedquestion database based on the search query to generate frequently-askedquestion search results; and outputting a display page, wherein thedisplay page categorizes the keyword search results and thefrequently-asked question search results into a plurality of categories,wherein each category is associated with a category title, a firstdisplay region, and a second display region that is separate from thefirst display region, wherein the second display region of a particularcategory includes one or more frequently-asked question search resultsassociated with the particular category and does not include any of thekeyword search results for the particular category; wherein astatistical cluster analysis is performed to determine the plurality ofcategories for a particular application based on group assignments andname assignments.
 13. The computer-readable storage device of claim 12,wherein a card sort method is used to perform the group assignments andthe name assignments.
 14. The computer-readable storage device of claim13, wherein the group assignments performed by a first set ofindividuals is validated via a reverse card sort method.
 15. Thecomputer-readable storage device of claim 13, wherein the operationsfurther comprise: determining a first category that best matches thesearch query and a second category; and displaying informationassociated with the first category in a row separate from informationassociated with the second category.
 16. A system, comprising: aprocessor; and a memory including computer program code executable bythe processor to: receive a search query from an input field of a userinterface; perform a keyword search based on the search query togenerate keyword search results; perform a natural language search of afrequently-asked question database based on the search query to generatefrequently-asked question search results; and output a display page,wherein the display page categorizes the keyword search results and thefrequently-asked question search results into a plurality of categories,and wherein the display page includes: a first region including a firstlisting of one or more of the keyword search results associated with acategory, wherein the first listing does not include any of thefrequently-asked question search results; and a second region includinga second listing of one or more of the frequently-asked question searchresults associated with the category, wherein the second listing doesnot include any of the keyword search results; wherein a statisticalcluster analysis is performed to determine the plurality of categoriesbased on group assignments and name assignments.
 17. The system of claim16, wherein a card sort method is used to perform the group assignmentsand the name assignments.
 18. The system of claim 17, wherein the groupassignments performed by a first set of individuals is validated via areverse card sort method.
 19. The system of claim 16, wherein the memoryincludes computer program code executable by the processor to output asecond display page that displays additional keyword search results andadditional frequently-asked question search results in response toselection of a user-selectable option.
 20. The system of claim 19,wherein a frequently-asked question search result associated with aparticular category is shown in a first row of the second columnassociated with the particular category when the frequently-askedquestion search result best matches the search query compared to otherfrequently-asked question search results associated with the particularcategory.